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ABSTEACT 

The mission, organization, rationale, roles, and 
functions of the department are discussed. The status of the 
d^piitmen and its interface^ vith other departments are also 
described. The paper deals with problems of funding, credibility', 
illations between research and evaluation versus planjiihg and 
development, and problems of relating research and devi^lopiient to 
other departments of the school system. (Author/HLF) 



U S 06PA«TMENT OF HEALTH, 

EOOCATION* WELFA«E \ 
NATIONALtf«STlTUTE OF 
^ EDUCATION 

?:j^^p°e4dto;G/:As./*^.iNu«^ organization Atro administration of the 

■ «luo o?^o?^°«V»^A«^" "'epvI:' dep artment of research and DF.VELOPMENT 

^ ll^'c?trosPor.:^oi;:^J,c^"^ ~ Th the Cincinnati public schools*' 
o- 

Q By: Dr* James N. Jacobs, -Dr. Gerald L. Varland 

; . _ 

ORGANIZATION _n 

' The Department of Research and Development is one of four 
departments headed by Assistant Superintendents constituting th.e 
basic organization of the Cincinnati Public Schools (see Organization 
Chart). The Department consists of five branches whose common 
linkage is that of delineating, obtaining and providing relevant 
information to staff and other personnel to- help them improve the qualilfy 
of- educational decision making. While each pranch has this common 
thrust, each deals with different types of ^information and/or with 

/ ^ 

' different emphasis* I 

The Department of Research and Deveioj)ment was constituted in 
1972 as an analgamation of three branches; Program Research and 
Design; Testing Services (these two branches were reporting to a 
Department Head in charge of Instruction); and Administrative ResearcK 
(which reported directly to the Superintendent), This new organi-^ 
ration came in with a new Superintendent:. 



5 ;/ 



/ 



ERIC 



Branch Descriptions, 

the Administrative Research Bianch was organized in the early 
1900*8 .to_provide the Superintendent, the Board of Education and the 
State Department .of Education with bread-and-butter educational 
statistics. It still has that function. It collects and provides 
data on: school attendance zones, actual and projected membership, 
<i salary schedules, class sizeU State funding of schools, school building 

9^- * Paper presented at A.E.R.A., Division li, April 1975 
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utilization, per-pupil costs, determination of school levies needed, 
analysis* of votes on levies, to mention only a few. 

The Administrative Research Branch has two other broad functions 
that distinguish it from the remaining branches • First, it is the 
branch which has stewardship for Board of Education policies* In this 
regard this branch identifies areas of policies for which none have 
been written and calls them to the attention of the Board of Education 
It also collects and classifies present Board policies into a coherent 
scheme which makes it readily useable by members of the Board of 
Education. The second function is that of developing and tnaintaxning 
a school unit information system. The latter is an elaborate data 
base providing several hundred descriptor variables for each school 
in the school system that assists administrators, planners and develop 
ers in dacis ion-making. The information system produces eleven 
separate school reports -to principals each year. Included among these 
are: , exceptional characteristics report, achievement expectancy 
report, basic data report, form survey reports (teacher, student, 
parent and administrator) trend reports, ecc.^ 

The Testing Services Branch has the mission of planning and 
implementing the program of standardized testing for the school 
system. This year for the first titae the 'testing program will 
consist of continuous grade testing from grade one through grade 
eight in the basic skills of reading and arithmetic. In addition, 
this branch administers scholastic aptitude tests in grades three, 
six and eight. Within the next three years, the Testing Services 
Branch will be responsible for administering a criterion-referenced 
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Inventory both in, reading and arithmetic which is now Being developed 
In the Planning Development Branch* In addition to the respon** 
sibillty for standardized testing, the branch also provides training 
and consulting services for the teaching staff on matters pertaining, 
to, test interpretation, and selection and use of tests. It also . 
provides testing services for special programs such as the G^EH). and 
selection tests for Special College Preparatory Programs operated 
by the school system. 

The mission of the Planning and Development Branch is to identify 
priority educational needs of the school system and to plan programs 
to meet these needs. In accomplishing thisjnission , the branch is T 
deepl5^ involved in needs assessment and goal development at both school 
system and local school levels. This branch utilizes information f 5pm . 
other branches from Research and Development which reveal educational 
ne^ds and helps order them into priority goals; Programs are then 
evaluated or developed which bear on these priority goals. If / 
promising program strategies are not available to meet^ these goals, 
they are designed by members of this branch and usually submitted as 
proposals to various funding agencies. Perhaps as much as thirty to 
forty percent of the time of this branch is spent on proposal develop- 
ment. In this process extensive work is done with community groups 
and staff Ito' insure responsiveness of the proposal to the identified 
need. This branch also obtains information on all types of resources, 
grants or otherwise, that can be used for program development. In 

many cases, technical assistance is given to non-members of the Research 

» 

and Development staff in developing proposals. 



This branch constructs student, teacher, parent and administrator 
surveys each year to identify problems and assess the needs of the school 
system. Over the past couple of jrears, the Planning and Development 
Branch has gotten deeply into computer applications in education ^nd 
has acquired a- rather , substantial, well-rtrained staff. Since we 
consider computer applications to education as being a continuous 
process, it is likely that this unit will remain in the Planning and 
Development Branch and not be mainstreamed per se. 

Program Evaluation Branch . Using terminology from the CIPP 
evaluation model, the three previous branches may be thought of as 
contributing to context evaluation, with the Planning and Develop- 
ment Branch emphasizing input evaluation. In contrast, the Program 
.Evaluation Branch emphasizes process and product evaluation. Pout- 
types of evaluation roles are defined: project evaluation, indepen- 
dent program evaluation, external evaluation and local school evalu-- 
ation. The pro;iect evaluator, is assigned exclusively to a specific 
project and reports to the project director. The job is primarily 
formative (internal) evaluation, and is conducted during the first , 
year or two of the project^s development. As the project matures 
and finished products or specific outcomes are produced, we phase 
out the project evaluator and phase in an independent program eValuator , 
We call this role independent because the evaluator reports not to 
the project director but rather to the coordinator of the Program 
Evaluation Branch. The independent program evaluator focuses attention 
on outcomes and overall impact (goal-free) and makes' recommendations 



on recycling decisions. The third role of external evaluator also 
focuses mainly on summative evaluation but for projects operated 
outside of Research and Development, particularly Title I and other 
State-funded projects* These evaluations are strictly goal-based/ 

To our knowledge, Cincinnati has. developed and pioneered a unique 
evaluation concept which we call Local School Evaluation (LSE) • 
In contrast to other types of evaluators, LSE^s are thought of as 
adjuntft members of local school staffs* Their inost distinguishing 
characteristic is their responsiveness to felt needs identified by 
the principal or school staff. Their activities range from identifying 
needs to measuring outcomes. They might be called upon to perform 
survey research, a literature review on learning styles, an item 
analysis on a test, or a project evaluation* A major goal is to -draw 
local school staff into a way of thinking about improvement of edu*-- 

cation (the R and D way?)* 

r . ' 

Under certain clrcurastartces we have also hired external • auditors ^ 
when credibility may be a sensitive issue. This auditor is expected 
t'o bring three major qualities to the project operation: objectivity, 
expertise, and perspective. The auditor, through a planned series 
*of activities, determines the appropriateness of evaluation in design 
and operation. The auditor verifies the results of the evaluation 
process, and adds an additional measure of credibility. A project 
auditor is- used only in projects where the funding agency supports 
the service,. The auditor reports his findings directly to the funding 
source after consultation with project staff. 



A paper presented by -Bresnan, Varland and Gastright^^^ goes into 
much more detail on types' of evaluation- roles and their respective 
strengths and weaknesses.' It should be pointed out that the Program 
Evaluation Branch staff may- not be exclusively assigned .one of the four 
aforementioned roles. A given/ person may be assigned 50% project 
evaluator and 50% external evaluator. Assignments change from year 
,to year and even month to month, depending on current conditions. 
Further, one of these roles may hot be actually fulfilled at^a given 
time. ' . 

Finally the Program Evaluation Branch administers a^ Professional 
Library for, the total school system and is largely responsttle for , _ 
R and J)^5publications. 

Project Management Branch . Whereas the 'Planning. and Development 
Branch develops project proposals, and whereas the Program Evaluatiot/ 
Branch conducts the evaluation of the projects, the Project Management 
Branch is responsible for project implementation. This branch phases- 
in new projects, assists in hiring and training staff, serves as 
liaison between the project proposal planners and the project staff, 
supervises project directors, manages the budget and, upon termination 
cf the grant, phases the project out of Research and Development and 
assists in its institutionalization in the mainstream of the school 
system. This branch plays a trouble-shooter role in the host school(s) 
where the project is being implemented. It also phases-in involvement 
of line organization personnel so as to obtain their assistance, support, 
' and commitment which is essential to the final goal of program 
institutionalization. 



Bresnan, Varland, Gastright. Spectrum of Objectivity-Cred/bility 
in Evaluation . Paper presented at A.E.R.A. , April '1975t / 
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Intra-Departmental Coordinati on 

Because of the high degree of inter-face required among Research 
and Development branches, weekly meetings of the R and D Council are* 
held. The Council Is composed of the five R and D branch coordinators 
plus one elementary and one secondary principal^ representative from 
their respective organizations* The principals keep us "reality Based 
and form a communication linkage v;ith school administrators. Other 
personnel participate in Council meetings on invitation, depending 
on agenda topics. The Council is chaired by the Department Head. • 

The Council serves: / 

/ 

1. To identify goals and establish priorities for the / 
Department and' to facilit^ite the'ir achievement 

2. To provide developmental and operational guidelines ^ 
for Department activities 

3. To hear and evaluate progress reports from various 
projects or activities within branches 

4. To assign work to the various branches in response 
to the needs and/or requests of the Department 

5- To provide and- to disseminate information within and • 
outside the Department 

6. To coordinate activities of the R and D Department 
with other units of the school system 

7. To uncover and try to resolve problems that impede 
departmental goal attainment 

8. To coordinate the activities with the R and D Depart*- 
ment. J 

Inter-Departmental Coordination and Inter-Face 

This function is handled through the Cabinet which is headed 
by the Deputy Superintendent and is composed of the four Department 



Heads* The Cabinet functions for the .four departments as R and D 
Council does for the branches within Research and Development. While 
most of _^e information flowing from Research and Development is utilized 
by the Department of Curriculum and Instruction (C & I) , a substantial 
portion is also direct jad to the Departments of Human Resources and 
Support Services^ Needs assessment services by Research and Develop-- 
-ment are probably more prominent than any others for the two latter 
departments. 

Inter-face activities between Research and Development and 



Curriculum and Instruction are c 



nature* Sensitivity arises from 



mmon and sometimes sensitive in 



the fact that Research and Develop- 
ment constructs programs and curricula which are often in competition ^ 
with' ongoing programs of the Department of Curriculum and Instruction. 
The ultimate goal of R and D proj.ects^^^of course, is to mainstream 
those- projects which have merit and to meet goals which were either" 
previously unmet or met better than the existing programs. Over the 
years we have learned that the transition from experimental .programs to 
mainstream programs occurs best v;hen members of the line organization 
(usually Instructional Consultants) are involved in all phases of 
the development process — from planning to implementation and. 

( . 

evaluation* Such involvement creates a feeling of ownership and 
commitment, such that upon termination of the project, the transition 
occurs rather smoothly. Often, in fact, a project staff from Research 
and Development will move right along with the project as it becomes 
institutionalized into the hard money line organization. We see this • 



.as an ideal situation. 
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The other most common linkage with Curriculum and Instruction* 
is the fact that^j^hey^re the decision-makers or the clientele to whom, 
most evaluation reports are targeted. Our evaluation personnel 
generally enjoy an atmosphere of respect and crediBilityv- When an 
eK^luation finding indicates bad news, it is discussed^ith the 
administrator on a face-to-face basis. Our recommendations are 
frequently jEoliowed although as you can imagine, many must be 
overlooked for a variety of reasons. 

ADMINISTRATION ^ 

Personnel * ^ 

At the time of writing, the^* Research and Development Department 

i- 

has thirty-fiye 'full-time professional staff members. This number 



varies greatly from year to year-, 'depending on the number 



of project 



grant awards. All but a few members of the staff have a Masters Degree 

•and about one^third\of them have Doctorates. Mo*^*" of them have, back- 

grounds in development and evaluation work, with the exception of our 

project directors who we often hire because of their specialized 

knowledge in a certain area. For example, the staff of -a preschooT 

' , . "* I 
project, including the project director, are specialists 'in that area. 

Keeping staff members in full-time emplojrment. poses a special 

problem. VJhen personnel are hired, they are made aware of the fact 

that their tenure in Research and Developmen.t is very likely limited 

to the grant period. Yet, the strength of the Department lies in its 

personnel, and it is' very difficult to release a person when it is 

known that he/she has excellent qualifications for R and D work. 

' \ 

■ \ 

\ 
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Furthertnore, the distinctions we make among specialists-^are not 

understood or appreciated by field practicio'iiefs.'^"'^ 

I 

Management By Objectives ' J 

Each year the Department sets forth a set of goals based on 

function, n^'ed and anticipated resources. This year we are operating 
/ 

under an? eighteen goal structure. The Research ^and Development 

:\ . "n^ ' ' ' 

Council W9rk*5 through a process of goal reduction until a set of 

activities or objectives is identified which can then, be assigned 

to every member of theXstaff . The Department Head sets forth the 

major goal areas and these are. reduced to subgoals by branch cobr-- 

dinators. They in \ aTti rejuce the subgoals into objectives and 

activities with respect to their branch staff. This whole process is 

one of negotiation and our experience to date has been that we list 

much more work to be done than we can practically do. We are ^ .ning ' 

a better appreciation, however, of what is and what is not realistic • 

T e process, which ends an the identification of work tasks, can be 

very laborious, but it is probably worth the effort. It, sets the basic 

mold for Department coordination and expectation. While tat^t dates 

are normally set for the completion of the objectiV^es. we have not ^- . 

gone very far in devising schemes for racing, the quality of the work ^ 

completed. We do know,, however, whether or not the jobvwas done* 

In addition to the work that was planned to be done, it is 

obvioup that much work is identified during the course of the year 

that was never anticipated.. The system of assigning unanticipated 

work generally consists of identification and assignment, of work 

within R and D Council. These assignments are recorded on "Work 
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Assignment forms and summarized by the Department head at the end of 
each year. 

Co-Mingling of Resources 

Virtually all projects require specialized resources such as 
media services, photography, editing, etc., whichf . difficult 

to obtain within the resources of a particular project. When we 
plan projects, however, we make each contribute a certain amount to 
the operation of these specialized services. We have, for example, ^ 
an elaborate media center operated by a top-flight specialist who' 
gives services to many projects and, of course, each project contribute* 
to its operation. The center provides photipgraphic service, graphics', 

Jayout and design work, audio-video tapes, etc. ^ They are capable 
of producing' a project brochure, set up a standardized test or 
publ^ish an annual report. ^ 

Similarly, we have a computer applications^ celiter which is 
funded by many projects, and which^^liow is capable of producing 
statistical analyses, project budget control, scoring surveys and 
producing all kinds of management reports, to name a few. 

V ^ These services require a critical mass of our Department 
resources for their continued operation. 

Coordination of R and D Activities Outside the department 

Like all large cities, we'are besieged with university students 
requesting permission to conduct research in our schools. For the 
most part, field personnel resent intrusion by students whose labors- 
find no practical application by school people. In defense, 



have developed a pol:^cy on research by outside persons or agencies 
which in effect requires that tlfe schools are to benefit in some 
meaningful way before permission is granted to conduct .the study. 

We have, however, acquired an offensive posture rather than a 
defensive one. We novj, see masters and doctoral students and uni- 
versity staffs as a positive force and adjunct staff 'in the conduct 
of research aifi(^ development activities in the school system. We„have 
a simple/ground rule -\ we must identify the problem and set the 
conditions under which the research and development will take place. 
The trick, then, is to match the needs with the interests of students or 
university staffs. The Department retains a problems file for students 
or staff to work through in order to make an appropriate match. So 
far, this process has worked fairly well, but we are not terribly 

■ ' ■ f ' ■- 

impressed with the willingness of local university commitment to assist 

iri-the study of real problems. \ . . 

i ' , 

Publications 

We have a theory that the R and D process looks something like 
an hourglass. The top portion^ represents the planning, development, 
research and evaluation -that goes into an educational program. The 
bottom'^ortion represents the extensive utilization of the R and D 
effort. The middle of the hourglass is the bottleneck. One may sge this 

/ ■ 

^as^the diffusion process. For the most part, we diffuse our 

programs and products through written publications. We represent 

-^t as a bottleneck because of the strong belief that the written 
word is usually poorly written to begin with, and. secondly, because 
it is probably not the most effective medium with which to reach 



overworked educators. We are slowly but surely learning to use 
other kinds of media, probably. the best of which is called "eyeball 
to eyeball." Neverth^eless, we publish quantities of materials, each 
targeted to different audiences. 

i 

^ Briefings is a pamphlet which is published once \ 

a month, and is sent to all members of the. professional staff • The 
articles represent summaries of R and D findings and current .topics 

of interest. , • t ^-j-' * ...'^ > * " »" 

Information Highlights is a one-page publication which focuses 
.on a single topic per issue, and highlights the results of some 
investigation. For example, one issue was devoted to our mMsurement 
of staff morale in the school system and showed the. trend over the 
past several years. It is targeted to the professional staff. 

Each Child usually focuses on project descriptions and evaluation 
in some depth. It is carefully written in l.mguage that can easily^ 
be understood by most persons. 

Journal of Research and Development pulls out all the stops, 
and reports studies comprehensively and in technical language* 
It is targeted to other research^ personnel and is distributed to 
Large City Research Directors and many university libraries and 
foundations. We may phase-out this Journal soon. 

The most common publications are called branch reports and these 

1 

are directed to different personnel, depending^ upon the report. It 
is from these reports that the briefer reports are generated. 

The ability to write in a clear apd concise manner is probably 
the greatest overall weakness. Look at this manuscript! ^ 



